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The Effectiveness of Library Instruction: Do Student Response Systems 
(Clickers) Enhance Learning? 


Abstract 

In the present study, we were interested in determining if library instruction would be more effective if 
personal response systems (clickers) were used during instruction. Furthermore we were interested in 
examining if students in a class could benefit from clicker technology even if they did not have access to a 
personal clicker. To examine these issues, we conducted 3 library instruction sessions: Session 1-half of the 
students were randomly assigned a clicker; Session 2-all students had individual clickers; and Session 
3-clickers were not used. Although half of the students in Session 1 did not have clickers, they were presented 
with all of the information, were aware of the clicker questions, and were presented with the graphs of 
responses. Students in all 3 sessions completed a pretest and posttest and difference scores were calculated 
such that positive numbers indicated higher scores. Overall, scores were significantly higher for students who 
had access to clickers. A comparison of specific clicker use showed that both the individual and group clicker 
sessions led to significantly higher difference scores. Results indicated that the benefits of clickers are not 
limited to individual access and group clicker use was as effective. Overall, these results confirm research 
supporting the integration of technology into classroom instruction. 

Dans cette etude, nous avons cherche a determiner si la formation en recherche documentaire etait plus 
efficace lorsqu’on utilisait des systemes de reponse personnelle (televoteur). De plus, nous voulions savoir si 
les etudiants en classe profiteraient de cette technologie meme s’ils n’avaient pas acces a un televoteur 
individuel. Pour ce faire, nous avons organise trois seances de formation en recherche documentaire. Pendant 
la premiere, nous avons distribue un televoteur a la moitie des etudiants choisis au hasard. Pendant la 
deuxieme seance, chaque etudiant disposait d’un televoteur. Au cours de la troisieme seance, aucun d’entre 
eux n’a utilise cet appareil. Meme si la moitie des etudiants n’avaient pas de televoteur au cours de la premiere 
seance, ils ont re$u toute l’information, ils connaissaient les questions auxquelles il fallait repondre avec le 
televoteur et les chercheurs leur ont presente les graphiques des reponses. Tous les etudiants ont passe un 
pretest et un post-test et les differences de resultats ont ete calculees afin que les chiflfes positifs indiquent des 
resultats plus eleves. Dans l’ensemble, les etudiants qui avaient acces au televoteur ont obtenu des resultats 
significativement plus eleves que les autres. La comparaison de l’utilisation specifique du televoteur a montre 
que les seances en groupe et individuelles au cours desquelles les etudiants utilisaient cet appareil entrainaient 
une difference significativement plus elevee en ce qui a trait aux resultats obtenus. Les resultats indiquent que 
les avantages du televoteur ne se limitent pas a faeces individuel et que futilisation de ce dispositif en groupe 
etait aussi efficace. Dans fensemble, ces resultats confirment la recherche qui appuie l’integration de la 
technologie dans l’enseignement en classe. 
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Over the years, various technologies have been used by instructors as teaching tools. 
Recently, however, more active learning tools that require input from both the student and the 
teacher have become popular in the classroom (Auras & Bix, 2007). These interactive tools are 
considered by many to enhance traditional lectures by increasing student engagement and 
learning (Cain & Robinson, 2008; Menon, et al., 2004). One such tool gaining popularity in 
postsecondary education is the audience response system (ARS) that is referred to by several 
names including student response systems (SRS), personal response systems (PRS), classroom 
performance systems (CPS), and the colloquial term clickers (Cain & Robinson, 2008; 
Premkumar & Coupal, 2008). When clickers are used in a lecture, all students are provided with 
a clicker and the instructor incorporates multiple choice questions into his or her lecture. These 
questions are typically presented to the students via Microsoft PowerPoint™ and students 
respond to these questions using their clicker. Immediately after all the responses are made, a 
graph showing the pattern of responses is presented. This technology allows instructors to gauge 
the knowledge of specific concepts and, if appropriate, expand on the different answer choices. 
Although clickers can be used in a variety of ways and can change the delivery mode of 
information (i.e., Beatty, Gerace, Leonard, & Dufresne, 2006; Draper & Brown, 2004; Mazur, 
1997), they are typically used as a tool to enhance, rather than replace, traditional lectures. In the 
current study, we were interested in whether the use of clickers improved learning, and we 
integrated clicker questions into a traditional lecture. 

Clickers add a “game approach” that may be more engaging than other interactive 
methods (Martyn, 2007) and add an element of fun to the learning situation (Hoffman & 
Goodwin, 2007; Homme, Asay, & Morgenstern, 2004). Teachers also benefit by receiving more 
immediate feedback on student comprehension, more “honest” student feedback because of the 
removal of student conformity which may occur when students respond by raising their hands, 
and, in many instances, student participation reaches 100% (Eggert, West, & Thomas, 2004; 
Stowell & Nelson, 2007). 

Response systems have been used for a variety of purposes (Cain & Robinson, 2008), 
with most research confirming benefits in the classroom, such as increased student engagement 
and interest, greater discussion and interactivity (Auras & Bix, 2007; Martyn, 2007), and positive 
learner perceptions (Cain & Robinson, 2008; Medina et al., 2008). The few studies assessing the 
use of clickers to improve students’ academic performance have produced mixed results. For 
instance, Morling, McAuliffe, and DiLorenzo (2008) found small, positive effects on exam 
scores for large introductory classes using clicker technology. In contrast, Morgan (2008) found 
that grades were lower for introductory classes using clickers, although not statistically 
significantly lower, whereas Stowell and Nelson (2007) reported no significant effect on quiz 
performance. 

Library Instruction 

In 2005, Julien (2005) reported that 48% of Canadian libraries indicated that 
informational technology had changed how information about library resources was delivered, 
and over 95% indicated that technology had changed the content of information. Librarians are 
required to teach information literacy skills across a variety of disciplines both in and out of the 
classroom. To effectively complete assignments, university students must understand how to use 
the library and the resources available to them. With the belief that response systems could 
increase information retention and hence the literacy skills gained from library instruction, 
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Instructional Services at Texas A&M University began using clickers in basic library sessions in 
the summer of 2006 (Hoffman & Goodwin, 2007). These researchers incorporated several 
clicker questions into traditional 50-minute lectures on library resources. Although clickers were 
used, the mode of presentation did not change. Prior to implementation, library instruction 
feedback forms indicated that students felt overwhelmed by material presented in the sessions, 
with some stating that they would probably forget most of the infonnation. After 
implementation, Hoffman and Goodwin reported that the use of clickers in classroom sessions 
resulted in fewer students reporting feeling overwhelmed, confused, or uncertain about retention 
of the information presented, and most reported that using clickers made the sessions fun. 

Although Hoffman and Goodwin (2007) reported positive results from applying response 
systems during library-related instruction, there are few hard data on how effective information 
literacy classes are in teaching library literacy skills using this method (Petersohn, 2008). 
Petersohn (2008) conducted a pilot study to assess the impact of response systems during library 
instruction on short-term retention. Forty-eight freshmen students from Communication or 
English Composition classes at Georgia State University’s Alpharetta Campus participated in a 
20-minute library instruction session. Participants were randomly selected to be in a clicker or a 
nonclicker session. Clicker groups were given classroom instruction in a computer lab. At the 
beginning of the session, a five-question pretest was posted on the participants’ computer screen. 
Students responded using individual keypads, with responses immediately graphed on the screen. 
A lesson followed a discussion of the responses. Feedback requiring students to use their 
assigned keypads was obtained at points throughout the session. Upon completion of the class 
the students answered a six-item on-screen posttest. For the nonclicker group, neither computer 
screens nor clickers were used. Paper pre- and posttests were completed, and student feedback 
during the lesson was obtained by a show of hands. Results indicated that both groups showed 
improved scores between the pre- and posttests. However, Petersohn (2008) reported statistically 
significant differences between the clicker and nonclicker groups; specifically the clicker group 
showed more improvement than the nonclicker group; however, generalization of these findings 
is limited because of the small number of participants. 

Current Study 

As previously indicated, the research evaluating student response systems has provided 
mixed results. Some researchers have reported positive results, but others did not find that 
clickers led to higher retention. Given these results, we were interested in further examining the 
effectiveness of clickers among university students participating in library instruction sessions 
with the expectation that the use of clickers would lead to higher retention. We were specifically 
interested in examining overall class retention when clickers were available to only half of the 
students in the classroom. We expected that class retention would be higher in sessions when 
either all or half of the students had clicker access and lower in sessions when clickers were not 
used. 

Method 

The purpose of the current study was to determine whether student response systems 
increase knowledge about information literacy. 
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Participants 

Two hundred undergraduate students enrolled at the University of New Brunswick 
participated in group library instruction sessions, 80 males (M age = 19.94, SD = 1.68) and 113 
females (M age = 22.92; SD = 8.53); 7 participants did not report their gender. A total of 132 
participants identified themselves as first year university students, and 68 reported that they were 
in their second, third, or fourth year of university. Although first year students have not typically 
declared a major, 57 reported that they intended to major in Arts; 18 intended to major in 
business; and 69 intended to major in science. 

Fifty-four of the 200 participants were engineering students; their data were collected 
during a regularly scheduled class, and thus all engineering students were in a combined session. 
The remaining 146 were psychology students recruited from the Department of Psychology 
Research Pool who received class credit for their participation. There were no differences in 
retention of the psychology and engineering students, as evidenced by a nonsignificant t test, 
*(194)= 1.71,/? = .09. 

When the Research Pool was used, students signed up to participate in small groups of 20 
to 30 students. The group sessions were randomly assigned, and several different session times 
were scheduled. The different types of sessions were scheduled such that, in a given week, there 
was one individual, one combined, and one control session. Sessions were scheduled on different 
days and at different times of day. Participants signed up to participate in a research project on 
library instruction but were not aware that different types of sessions were scheduled. Even if 
participants were aware that there were different sessions (i.e., through word of mouth), they 
would not have been able to predict which sessions (individual, combined, control) that they 
were signing up for. Across all participants, the preexisting knowledge about the library was 
examined and no group differences were found, F( 2, 193) = 2.9, p > .05. Three participants’ data 
were removed from all analyses because their standardized retention scores were more than three 
standard deviations lower than the average. 

Equipment 

Thirty Interwrite PRS Radio Frequency personal response systems were used by 
participants. 

Procedure 

All sessions were conducted during Fall 2008 and were held in a university classroom 
that was equipped with an overhead data projector. Microsoft PowerPoint™ was used by the 
instructor to aid in the presentation of relevant information. In total, seven sessions were 
conducted: one combined (Engineering), two combined (Psychology), two clicker (Psychology), 
and two control (Psychology) groups. The basic procedure was explained to the students when 
they entered the classroom. Students were informed that they would be presented with 
information about the services offered by the library and that, to assess their learning, we wanted 
them to complete two tests: a pretest to assess their existing knowledge and a posttest to assess 
what they learned during the session. They were given the opportunity to ask questions about the 
procedure. After this introduction, they completed an infonned consent form and the library 
knowledge pretest. The pretest was designed by a librarian to assess general knowledge about the 
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library and more specific knowledge about how to obtain specific types of information. The test 
included 10 multiple choice questions (e.g., Which of the following is a characteristic of a 
scholarly journal? A. Usually read by the general public; B. Articles include 
references/bibliographies; C. Report on current events; D. Usually published weekly). 

After completing the pretest, participants were presented with specific information about 
library services and how to conduct academic research. Information was presented in three 
different formats: 

1. Combined Groups (N = 90; 58 males, 32 females): Students entered the classroom and chose 
a seat. After all students had completed the pretest, clickers were randomly distributed to half 
of the students. Each student with a clicker was seated beside a student who was not assigned 
a clicker. Although not all of the students had access to a personal clicker, all questions and 
tallied results were presented to the entire class (the specific clicker questions that were 
presented focused on the information in the session; however, individual clicker responses 
were not recorded). Thus, students who did not have a clicker were also presented with the 
questions and responses and could potentially benefit from this information. 

2. Individual Groups (N =56; 10 males, 46 females): All of the participants had access to a 
clicker, and thus all participants personally answered all questions and could compare their 
clicker response with the responses of the other participants. 

3. No Clicker Groups (N = 48; 13 males, 35 females): These sessions served as control sessions. 
Although the same information was presented and the same questions were asked, clickers 
were not used. 

The length of the instruction sessions was approximately 45 minutes, and the information 
presented in the three formats was identical. After the instruction session, participants completed 
a posttest (which included the same 10 questions as the pretest) to assess whether the instruction 
sessions were effective. 


Results 

To determine the overall effectiveness of the library instruction sessions, data from all 
sessions (combined, individual, no clicker) were pooled and pre- and posttest scores were 
examined. As can be seen in Figure 1, pre- and posttest scores did not differ as a function of 
university faculty (arts, science, business); furthennore, specific university major did not 
influence test scores. Overall, an independent groups t test was conducted and there was a 
statistically significant difference in pre- and posttest scores, /(199) = 9.41 , p = .0001. As 
expected, pretest scores were lower (M= 5.24, SD = 1.64) than posttest scores (M= 6.31, SD = 
1.47). Regardless of faculty membership, scores on the posttests were higher than those on the 
pretests, indicating that the instruction served to increase knowledge. Given the fact that library 
instruction is commonly used at our university and librarians are often invited to give in-class 
instruction, these results are positive as they suggest that the instruction actually does improve 
general knowledge about academic research. 
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Figure 1. The pre- and posttest scores as a function of university faculty. The error bars represent 
standard errors of the mean. 


A 2 (University Level: upper vs. lower) x 2 (Gender: male vs. female) x 2 (Test: pre vs. 
post) mixed model ANOVA was conducted to determine specific differences in learning. Results 
indicated that those in upper and lower level university courses had almost identical pre- and 
posttest scores, F(\ , 190) = .01, p = .92, suggesting that students at all levels can benefit from 
instruction. Interestingly, there was a statistically significant main effect for gender, F(\ , 190) = 
7.21, p = .008. Overall, females had higher scores than males (Mf ema ie = 6.02, SEM = .13; M ma i e = 
5.47, SEM = .16). The interaction between test type and gender approached statistical 
significance, F{ 1, 190) = 2.77, p = .09. On the pretest, the scores of males and females were 
almost identical, but on the posttest, the scores of female participants were higher, which 
suggests that library instruction was particularly effective for female participants. 

The Effectiveness of Student Response Systems 

A difference score between the pre- and posttest scores was calculated such that positive 
numbers indicated that the posttest score was higher than the pretest scores. Overall, the average 
difference score was 1.07 (SD = 1.61), indicating that participants answered approximately one 
more question correctly on the posttest. Although this is a modest gain, it does represent more 
than a 10 % increase in scores from the pretest to the posttest. 

In assessing differences in learning according to instruction type, an ANOVA indicated a 
statistically significant effect for instruction type, F( 2, 193) = 4.41,/? = .013. When participants 
had individual access to a clicker, the average difference score was highest (M - 1.63, SEM - 
.20). Difference scores were lower in combined groups in which half of the participants had 
clicker access (M= 1.07, SEM= .15) and in control groups in which clickers were not used (M = 
.81, SEM = .21). A Boneferroni post hoc test indicated that there were statistically significant 
differences between each of the instruction types, suggesting that the library instruction sessions 
that provided individual clickers were more effective than those in which only some students had 
clickers. Sessions that did not provide clicker access were the least effective. 

The three groups were not equivalent in tenns of gender composition; the individual 
clicker group included 10 males and 46 females, the combined clicker group included 58 males 
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and 32 females, and the control group included 13 males and 35 females. Because of these 
unequal numbers of males and females, an overall analysis of gender by instruction type would 
not have been statistically valid and, thus, was not conducted. 

To examine any possible gender differences as a function of overall instruction type, 
clicker use was coded such that participants who had individual access to a clicker were 
differentiated from those who had no clicker access. Thus, across all three groups (individual, 
control, and combined), participants who actually had a clicker were differentiated from those 
who did not. A 2 (Gender) x 2 (Instruction Type: clicker vs. no clicker) ANOVA was conducted, 
and the main effect for clicker use approached statistical significance, F( 1, 187) = 3.24, p = .07. 
When clickers were used, the difference between the pre- and posttest scores was, on average, 
1.33. When clickers were not used, the average difference between the pre- and posttest was only 
.94. 

The main effect for gender was not statistically significant, and there was no difference in 
the retention scores of males and females (M ma i es = .99; Mf ema \ es = 1-29). There was a statistically 
significant interaction between gender and instruction type, F(l, 187) = 4.68, p = .03. As can be 
seen in Figure 2, when clickers were not used, the difference scores for males and females were 
similar, but when clickers were used, the difference scores for females were significantly higher. 
Thus, the integration of clickers did not affect the retention of information for males but led to 
higher retention for females. 


■ Females 
Males 



No clicker Clicker 

Instruction Type 


Figure 2. The difference between pre- and posttest scores as a function of gender and instruction 
type. The error bars represent standard error of the means. 

Discussion 

There is a growing body of research suggesting that integrating technology into the 
classroom is effective and can lead to higher exam scores (Morling et al., 2008). We were 
interested in determining if library instruction would be more effective if personal response 
systems ( clickers ) were used during instruction. Our results suggest that library instruction does 
enhance student knowledge and, regardless of whether clickers were used or not, student scores 
on the posttest were higher than those on the pretest. Given that library instruction is widely 
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available and used to teach students about how to conduct academic research, this result is 
positive. The results of the current study suggest that student response systems further enhance 
student learning. When individual clickers were used, student learning was higher when all the 
students had clickers than when only half of the students had clicker access. Learning was lowest 
when none of the students had access to clickers. Although the gains in knowledge from pretest 
to posttest were modest, clicker use did appear to enhance learning. 

Figure 2 presents one of the most interesting findings. Regardless of whether they were 
in a combined or individual session, the retention scores of the females were higher when they 
actually had access to a clicker. Interestingly, the scores of the male participants were similar 
regardless of whether they had clicker access. Although most of the published studies on gender 
differences in clicker use focus on perceptions, Reay, Li, and Bao (2008) used a pretest/posttest 
model to examine the actual retention of information and reported that when no clickers were 
used, males had higher posttest scores, indicating greater learning. However, when clickers were 
used, there were no differences between male and female scores, suggesting that clickers reduced 
the gap in performance between males and females. Given that the results of Reay et al. (2008) 
differ from the current results, future researchers should closely examine possible gender 
differences associated with clicker use. 

Limitations 

Although clicker use did lead to increased knowledge, the gains were modest. On 
average, student scores were approximately 10% higher on the posttest as compared to the 
pretest. All participants were volunteers, and the information presented to them during the 
instruction sessions was not included on any of their formal tests or exams. It is possible that the 
participants were not optimally motivated to learn the presented information, and it follows that 
the gains associated with clicker use may have been different if retention were tested more 
formally. In the future, we plan to integrate the information presented in the library instruction 
sessions with specific course material and encourage instructors to include test questions specific 
to library information. This integration would allow us to examine the effectiveness of clickers in 
situations where students had more to gain from the instruction sessions. 

There is also need to address the fact that the participants were not randomly assigned to 
the various groups, and so differences as a result of group assignment or volunteering to 
participate in a given group may have caused the differences between groups. 

To assess the effectiveness of clickers, we employed a pretest/posttest model. Although 
there are methodological strengths associated with this design, in the current study there are two 
flaws. First, as is typical, the pretest and the posttest had the same questions. Second, the posttest 
was completed immediately following the 45-minute instruction session. It is possible that the 
length of time between the pre- and posttests was too short and participants were able to 
remember how they answered specific pretest questions. In retrospect, we should have had 
participants complete the posttest several weeks after the instruction sessions. This modification 
would tap into not only short-term retention but also any long-term benefits of using student 
response systems to enhance learning. 
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General Conclusions 

A comparison of specific clicker use showed that both the individual and group clicker 
sessions led to significantly higher difference scores. Results indicated that the benefits of 
clickers are not limited to individual access and group clicker use was as effective. Although the 
current results support the integration of clickers into the classroom, they also suggest that the 
gains associated with clicker use are modest. As noted by Morgan (2008), it is possible that how 
an instructor makes use of the clickers, rather the clickers themselves, determines effectiveness. 
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